1. Technical Field
The invention relates to systems for assessing the value of content. More particularly, the invention relates to systems for reliably evaluating large amounts of content in a distributed manner.
2. Description of the Prior Art
Many sites found on the World Wide Web allow users to evaluate content found within the site. For example, the Amazon® web site (www.amazon.com) allows users to submit reviews of books listed for sale, including a zero to five star rating. The Slashdot Web site (www.slashdot.org) allows users to “mod” comments recently posted by other users. Based on this information obtained from the users, the system determines a numerical score for each comment ranging from 1 to 5.
Because such systems do empower a great number of users to evaluate content, the scope and extent of the content that may be evaluated is great. However, because there is no restriction on the users that may participate, the reliability of the ratings is correspondingly diminished. In an effort to address this deficiency, such systems often allow users to evaluate the evaluations themselves. For example, Amazon® allows other users to evaluate the submitted reviews by indicating that they found a review helpful. Slashdot allows users to annotate submitted comments with attributes, such as funny or informative. The large number of submitted comments can then be filtered based on these annotations and the numerical score described above. Nonetheless, each of these approaches essentially relies on a mass consensus in which each contributor to the evaluation process is granted equal significance.
However, evaluation systems that adopt a more centralized, more controlled approach, e.g. commissioning a small number of trusted evaluators or editors, are inevitably overwhelmed by the immensity of the content in need of evaluation. Thus, while the reliability of the evaluations may increase, time constraints ensure that the scope and extent of the content evaluated is diminished.
Thus, there is a need for a new system of evaluating content that obviates this apparent tradeoff. Preferably, the evaluation system should be distributed in nature, ensuring that an extremely large amount of content can be evaluated without unduly burdening any individual evaluator. However, the distribution of the evaluation effort should be performed in a manner that preserves the integrity of the evaluation process. The evaluation system should thus provide evaluations for extensive content in a reliable manner.